• Create a PDF document and a website to communicate your own analysis
• Including your own text, analysis, table and chart
• Using data from an external source (a CSV file)
• Understand the data science workflow in R
• Gain confidence in R

• Designed for data science
• A lingua franca of social science
• Easy to make professional outputs (tables, charts, maps) to promote FGV/CEPESP
• ANY question you have has already been answered online

1 - Use RStudio for everything. R is the engine; RStudio is the Interface. If you do half your data cleaning in Excel, there will be no record of it and we won't be able to fix mistakes.

2 - Our work is in the form of a 'Recipe Book': A step-by-step guide to the data inputs (the ingredients), our analysis (the cooking instructions) and the outputs (the picture of the perfect meal).

3 - Use R Markdown (.rmd) files to combine analysis and outputs: R markdown files allow us to combine data processing in R - in clearly-defined 'chunks' - with the display of text, tables, charts and maps.

4 - Output is produced only when we press 'Knit': It is NOT an interactive playground like excel (though it can be with Ctrl+Enter).

5 - Our work should be self-explanatory and reproducible: Anybody with R should be able to open our work, press 'Knit' and produce the same outputs.

6 - Organize your work in Projects in R: For each major analysis, it's best to choose 'File' -> 'New Project' -> 'New Directory' from Rstudio. Save all your data inputs and outputs in this folder (which Rstudio will do automatically).

7 - Data frames (tables) are the main building block of our analysis: We focus on manipulating and visualizing tables of data, as these are the best way of organizing our data.

8 - Use meaningful names in your work: 'data_v1b_2_061215' won't mean anything in 3 months! All files and objects should reflect their role in the analysis.

9 -Process our data in a 'tidy' way: This means we will use a set of compatible 'packages' called the 'tidyverse' to make our analysis transparent and avoid common problems.

• Text vs. Code Chunks
  • Type text directly
  • Insert -> 'R' creates a code chunk
  • Chunks contain data processing and outputs (tables, charts etc.)
  • Use a separate chunk for each output

• Formatting Text
  • Cheat Sheet
  • Equations: In latex format, eg. $$ a^2 + b^2 = c^2 $$

\[ a^2 + b^2 = c^2 \]

• Data analysis within code chunks:
  • Assigning to saved objects: new_object <- old_object
  • Inspecting objects interactively: Type their name and press 'Ctrl-Enter'
  • Processing objects: data_frame %>% action_on_dataframe
  • Comments: #Comments go here and won't be processed by R
  • The actions (functions) we can use depend on the packages we have loaded:
    • install.packages("New_package") ONCE, then
    • library("New_package") at the start of each document

• Basic Maths in code chunks:

answer <- 2 + 2
answer

## [1] 4

inputs <- seq(0,1,0.2)

answer <- inputs*10
answer

## [1]  0  2  4  6  8 10

• Once we have all the text and chunk outputs ready, Knit!
  • Knit to PDF
  • Knit to HTML

• Set the title, date, author etc. in the header in markdown

• Remember, we can produce Documents or Presentations
  • When we first create a new markdown file

1. Dataframe - A table from a file - read_csv
2. Process data
   • Cleaning - filter, select, mutate
   • Combine datasets - left_join
   • Create measures/statistics - mutate, summarize
3. Run regression - zelig
4. Create results
   • Table - kable, stargazer
   • Graph - ggplot
   • Map - leaflet, mapview

• Specific packages let us access APIs for online data, eg.
• Or from local files using read_csv

data <- read_csv("data.csv")

• To open SPSS or Stata files:

library(foreign)
data <- read.spss("data.sav")
data <- read.dta("data.dta")

• Variable Names
• Observations
• Values and variable types

flights %>%
group_by(carrier) %>%
summarize(dep_delay=mean(dep_delay)) %>%
kable()

flights %>%
group_by(carrier) %>%
summarize(dep_delay=mean(dep_delay)) %>%
ggplot() + geom_point(aes(x=air_time,y=dep_delay))

flights %>%
group_by(carrier) %>%
summarize(dep_delay=mean(dep_delay)) %>%
zelig(dep_delay ~ carrier,data=.,model="ls") %>%
stargazer(digits=3)

• select specific variables (columns)
• slice observations (rows)
• filter observations (rows) by conditions (based on values in columns)
• count number of observations
• rename variables (columns)
• arrange table in the order of a particular variable
• mutate (change) values of an existing or create a new variable
• summarize data by creating statistics
• round values to a specifc number of decimal places

flights %>% 
  select(carrier,origin,air_time,distance,dep_delay) 

flights %>% 
  select(carrier,origin,air_time,distance,dep_delay) 

## # A tibble: 5 x 1
##   air_time
##      <dbl>
## 1      227
## 2      227
## 3      160
## 4      183
## 5      116

flights %>% 
  slice(1:2)

flights %>% 
  slice(1:2)

## # A tibble: 2 x 5
##   carrier origin air_time distance dep_delay
##     <chr>  <chr>    <dbl>    <dbl>     <dbl>
## 1      UA    EWR      227     1400         2
## 2      UA    LGA      227     1416         4

flights %>% 
  filter(origin=="JFK")

flights %>% 
  filter(origin=="JFK")

## # A tibble: 2 x 5
##   carrier origin air_time distance dep_delay
##     <chr>  <chr>    <dbl>    <dbl>     <dbl>
## 1      AA    JFK      160     1089         2
## 2      B6    JFK      183     1576        -1

flights %>% 
  mutate(air_time=round(air_time/60,3))

flights %>% 
  mutate(air_time=round(air_time/60,3))

## # A tibble: 5 x 5
##   carrier origin air_time distance dep_delay
##     <chr>  <chr>    <dbl>    <dbl>     <dbl>
## 1      UA    EWR    3.783     1400         2
## 2      UA    LGA    3.783     1416         4
## 3      AA    JFK    2.667     1089         2
## 4      B6    JFK    3.050     1576        -1
## 5      DL    LGA    1.933      762        -6

flights %>% 
  mutate(speed=round(distance/air_time,3))

flights %>% 
  mutate(speed=round(distance/air_time,3))

## # A tibble: 5 x 6
##   carrier origin air_time distance dep_delay speed
##     <chr>  <chr>    <dbl>    <dbl>     <dbl> <dbl>
## 1      UA    EWR      227     1400         2 6.167
## 2      UA    LGA      227     1416         4 6.238
## 3      AA    JFK      160     1089         2 6.806
## 4      B6    JFK      183     1576        -1 8.612
## 5      DL    LGA      116      762        -6 6.569

flights %>% 
  summarize(avg_distance=mean(distance,na.rm=TRUE))

flights %>% 
  summarize(avg_distance=mean(distance,na.rm=TRUE))

## # A tibble: 1 x 1
##   avg_distance
##          <dbl>
## 1       1248.6

These actions can be 'piped' together:
We want to find the average speed of United (UA) flights.

In steps: Take the data, filter the data to carrier UA,
calculate the speed of each flight,
and then find the average.

flights %>% filter(carrier=="UA") %>% mutate(speed=distance/(air_time/60)) %>% summarize(avg_speed=mean(speed,na.rm=TRUE)) %>% round(1)

flights %>% filter(carrier=="UA") %>% 
  mutate(speed=distance/(air_time/60)) %>% 
  summarize(avg_speed=mean(speed,na.rm=TRUE)) %>%
  round(1) %>% as.numeric()

## [1] 420.9

These actions can be 'piped' together:
We want to find the average speed of United (UA) flights.

In steps: Take the data, filter the data to carrier UA,
calculate the speed of each flight,
and then find the average.

avg_speed <- flights %>% filter(carrier=="UA") %>% 
  mutate(speed=distance/(air_time/60)) %>% 
  summarize(avg_speed=mean(speed,na.rm=TRUE)) %>%
  round(1)

The average speed of United Flights is `r avg_speed` miles per hour.

The average speed of United Flights is 420.9 miles per hour.

Can we change the order of data processing?

In steps: Take the data, calculate the speed of each flight,
filter the data to carrier UA,
and then find the average.

flights %>% mutate(speed=distance/(air_time/60)) %>% filter(carrier=="UA") %>% summarize(avg_speed=mean(speed,na.rm=TRUE)) %>%
round(1)

Can we change the order of data processing?

In steps: Take the data, calculate the speed of each flight,
filter the data to carrier UA,
and then find the average.

flights %>% mutate(speed=distance/(air_time/60)) %>% filter(carrier=="UA") %>% summarize(avg_speed=mean(speed,na.rm=TRUE)) %>%
round(1)

420.9

Can we change the order of data processing?

In steps: Take the data, calculate the speed of each flight,
find the average,
and then filter the data to carrier UA,

flights %>% mutate(speed=distance/(air_time/60)) %>% summarize(avg_speed=mean(speed,na.rm=TRUE)) %>% filter(carrier=="UA") %>%
round(1)

Can we change the order of data processing?

In steps: Take the data, calculate the speed of each flight,
find the average,
and then filter the data to carrier UA,

flights %>% mutate(speed=distance/(air_time/60)) %>% summarize(avg_speed=mean(speed,na.rm=TRUE)) %>% filter(carrier=="UA") %>%
round(1)

394.3

flights %>% slice(1:5) %>% 
  select(carrier,origin,air_time,distance,dep_delay) %>%
  kable()

flights %>% slice(1:5) %>% 
  select(carrier,origin,air_time,distance,dep_delay) %>%
  kable(caption="Example Table", align="lcccc")

flights %>%
  filter(carrier=="UA") %>%
  ggplot() + geom_point(aes(x=dep_time,y=dep_delay))

flights %>%
  filter(carrier=="UA") %>%
  ggplot() + geom_point(aes(x=dep_time,y=dep_delay)) +
  geom_smooth(aes(x=dep_time,y=dep_delay))

flights %>%
  filter(carrier=="UA") %>%
  ggplot() + geom_point(aes(x=dep_time,y=dep_delay)) +
  geom_smooth(aes(x=dep_time,y=dep_delay)) +
  ggtitle("Example Chart") +
  xlab("Departure Time") +
  ylab("Departure Delay")

flights %>%
  ggplot() + geom_bar(aes(x=dep_delay))

flights %>%
  ggplot() + geom_bar(aes(x=dep_delay)) +
  xlim(-30,100)

flights %>%
  group_by(origin) %>%
  summarize(avg_delay=mean(dep_delay,na.rm=TRUE)) %>%
  ggplot() + geom_col(aes(x=origin, y=avg_delay))